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Article history: The aquaculture industry has rapidly grown over the year. One pertinent 
aspect is the ability of the aquaculture farm management to accurately count 
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Revised Jul 18, 2023 density. The current practice of counting the fish manually increased the 
Accepted Sep 8, 2023 hatchery workers workload and led to inefficiency. The presented work 
proposed an intelligent, web-based fish counting system to assist hatchery 
workers in counting fish from images. The methodology consists of two 
Keywords: phases. First, an intelligent fish counting engine is developed. The captured 
image was first enhanced using the contrast limited adaptive histogram 
equalization. A deep learning architecture in the form of you only look once 
(YOLO)v5 is used to generate a model to identify and count fish on the 
image. Second, a web-based application is developed to implement the 
Usability test developed fish counting engine. When applied to the test data, the developed 
You only look once engine recorded a precision of 98.7% and a recall of 65.5%. The system is 
also evaluated by hatchery workers in the University Malaysia Sabah fish 
hatchery. The results of the usability and functionality evaluations indicate 
that the system is acceptable, with some future work suggested based on the 
feedback received. 
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1. INTRODUCTION 

Analyzing information about the underwater environment is important for preventing marine threats, 
preserving the ocean ecosystem and resources [1]. In recent years, the aquaculture industry has expanded due 
to the adoption of marine ranching techniques. Various work that utilizes artificial intelligence in aquaculture 
can be found in the literature, which includes fish classification [2], water quality monitoring [3], fish 
behaviour detection [4], and feeding control [5]. Accurate counting of fish populations in fish farms is critical 
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for proper scientific feeding, sale, export, and control of breeding density [6], [7]. Whenever the number of 
fish decreases, it could indicate non-optimal conditions occurred at the farm and requires the farmers to 
check all the relevant parameters (e.g. the food source, temperature, and oxygen). A quick response to any 
issues pertaining to the decrease in the number of fish may ensure that production will not drop significantly. 

The handling of fish seed in fish farms is labour-intensive, and prone to disruption by many factors 
such as the weather and water clarity [8]. It may take eight workers to count approximately 5 million fish 
seeds in one day [9]. Challenges faced include variations of sizes of the fish and also the complexity of the 
background environment [10]. For example, in University Malaysia Sabah (UMS) fish hatchery, manual fish 
counting is still taking place. As a result, other activities such as fish feeding, and cleaning of the fish tanks 
may be delayed. A fish farm that utilizes an automated system to count fish from an underwater image may 
improve the effectiveness of its management [11], [12], which subsequently increases production. 

Underwater images often suffer from distortion and light attenuation due to the scattering of light as it 
passes through the water [13], [14]. Thus, image enhancement is important for accurately detecting the presence of 
fish in a single image. Once the visual presentation of the image is enhanced, the detection of fish is performed. 
Various techniques were used; the most common are blob detection and deep learning based. A number of work 
for automated fish identification and counting is presented in [15]-[17], but they are either tested in a research 
setting or developed as a standalone application, which limits access to the system. A deep learning architecture, 
you only look once (YOLO), has shown efficiency in object detection [18]. A fish counting system should deploy 
an effective technique for image enhancement and efficient fish detection to increase accuracy. 

This paper presents a deployed intelligent fish counting system that uses underwater images is 
presented. The contributions of this paper are: i) contrast limited adaptive histogram equalization (CLAHE) 
was used to improve the presentation of the images and YOLOvS to identify and count the fish in the images 
and ii) a web-based prototype was developed to test the intelligent fish counting system, whereby the usability 
evaluation of the prototype was conducted on hatchery workers at University Malaysia Sabah fish hatchery. 

This paper is organised as follows. Some background to the topics related to fish counting and the 
method proposed are described in section 2. In section 3, the results and analysis are presented. The 
conclusion and suggestion for future work are outlined in section 4. 


2. MATERIALS AND METHOD 
In this section, brief reviews of the materials used are described. The methodology used is also 
presented. 


2.1. Contrast limited histogram equalization 

Histogram equalization (HE) is a process that creates a new version of an image with a more 
balanced distribution of intensity values. This is achieved by rearranging the intensity values of the original 
image in order to expand the range of values in the image histogram. The CLAHE method is a variant of HE. 
It limits the range of intensity of the output image to prevent outliers in the histogram that avoids 
oversaturation of regions with high intensity values. 

To perform CLAHE, first, the histogram of the intensity values in the image is calculated. This is 
followed by the transformation of the image using a cumulative distribution function. CLAHE is a commonly 
used method for enhancing images [19], and more information about it can be found in [20]. 


2.2. YOLOvS5 

Unlike other complex object detection networks, YOLOv5 uses a single neural network to the image, 
then separates it into regions and predicts bounding boxes and confidence values for each region. Weight and 
biases are directly integrated into YOLOv5, providing experiment metric tracking, model and dataset 
versioning, and rich model prediction visualization. The class probabilities are used to weigh these bounding 
boxes. It requires only one forward propagation pass through the neural network to make predictions. It 
generates detected target objects together with the bounding boxes after non-max suppression. YOLOvS5 can 
predict several bounding boxes as well as their class probabilities. When training on images, the YOLOv5 
network learns generalizable representations of objects, outperforming existing detection approaches. 

YOLOv5S uses cross stage partial networks (CSPNet) as its backbone to extract informative features 
from an input image. It also minimizes model parameters and floating-point operations per second (FLOPS), 
resulting in improved model speed and accuracy while reducing the model size [21]. A path aggregation 
network (PANet) is applied as the model neck to reduce processing time. PANet employs a novel feature 
pyramid network (FPN) topology with an improved bottom-up approach to enhance the propagation of low- 
level features [21]. Feature pyramids aid models in generalizing when it comes to object scaling. Figure 1 
shows the YOLOv5 architecture employed in this paper [21]. A cross stage partial network (CSP) is the basis 
of the architecture. It has several convolutional layers (conv) and concatenate functions (concat). The pooling 
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is conducted using the spatial pyramid pooling (SPP). The input images are supplied through the backbone. 
The PANet was used to perform feature fusion. Finally, the detection was conducted at the YOLO layer. 
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Figure 1. The YOLOv5 architecture 


2.3. Convolutional neural network for fish counting 

A deep convolutional neural network (CNN) is used to extract and learn complex yet abstract features of 
the input video for object detection [22]. For object detection, object segmentation is needed to locate the target in 
the image extracts from the underwater video. The architecture is built to process images with 224x224 pixels, 
adequate for extracting useful features within reasonable computational time. It can capture fish of varying sizes 
that appear simultaneously in images. The method will select the region of interest (ROI) in the input image’s 
feature space to detect the target objects in the image based on their texture, size, and shape features. 

An automated fish counting approach based on a dilated convolution neural network (DCNN) model is 
presented to accomplish the real-time counting of the salmon population [17]. In the proposed model, convolution 
kernels of various sizes adapt to the fish movement changes in angle, shape, and length. It can minimize the loss of 
spatial structure information during network transmission. Regression methods are then used to learn the 
relationships between image features to solve counting in large-scale noisy scenes and generate a density map. 

Transfer learning is used to address the limited dataset issue in [23] for fish species classification. 
Google inception-v3 pre-trained model was selected. There are three different approaches that were 
examined in the work, including a pre-trained model approach that used Inception-v3 as the feature extractor 
and the support vector machine (SVM) as the classifier. This approach produced the best results. 

A blob-based analysis for fish counting can be found in [12], [24], [25]. The user is required to input 
an image from a computer folder into the system [12]. Using a MATLAB graphical user interface (GUD), the 
edge detection and blob analysis were conducted to identify the number of fish in the image. The fish counting 
model produced good accuracy when applied to a small, medium and large samples dataset. Based on the 
implementation presented, the system is developed as a standalone application, thus limiting the mobility of the 
system. In the actual settings of a fish hatchery, application mobility would be an added advantage. 

Although a number of similar works can be found in the literature, all the presented work focused 
more on the research point of view, whereby the main aim was to produce the fish counting model with the 
best accuracy. In the context of the proposed work presented in this paper, we evaluate the fish counting 
performance and also the usability of the system at the real fish hatchery to ensure the model is actually 
usable by the targeted users. To the best of our knowledge, none of the previous work has tried to measure 
how well the developed model could be used in a real fish hatchery setting. 


2.4. Method 

The methodology consists of four phases; i) requirement analysis, ii) development of the fish 
counting engine, iii) development of the prototype of the web-based system, and iv) usability testing (UT). In 
the first phase, a literature review on the existing fish counting system, and work is conducted to identify the 
necessary features for a fish counting system. An interview with the target user, in the context of this paper is 
the UMS IPMB’s fish hatchery, is conducted and analyzed. The output of this phase is the required 
specifications of an automated fish counting system. 
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In the second phase, the intelligent fish counting engine is designed and developed. Figure 2 shows 
the framework of the model generation of the fish counting engine. First, the image is captured and enhanced 
using CLAHE. Next, the enhanced image is fed to the YOLOv5 architecture to generate the fish 


identification and counting model. The model generation is repeated several times using different epochs to 
identify the best model. The output of this phase is the fish counting model. 
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Figure 2. The flowchart of the fish counting model generation 


In the third phase, the design and development of the web-based automatic fish counting system 
were performed. Figure 3 shows the use case diagram of the proposed system. The user can update and even 
delete their profiles if they want to. The automatic fish counting function requires an image to be inserted 
into the system, either using the user’s phone camera or selecting an image from the existing repository. 
Once the image is inserted, the fish counting is performed, and the result is displayed to the user and stored in 
the database. The user can view their past fish counting activity through the view past result module. The 
prototype of the proposed intelligent system is the output of this phase. 

In the fourth phase, UT was conducted to get the users feedback on the automatic fish counting 


system. Questionnaires were distributed to the target users to get their feedback on the usability of the 
developed automatic fish counting system. 
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Figure 3. Use case diagram of the fish counting system 


3. RESULT AND DISCUSSION 


In this section, the outcomes of the requirement gathering, the implementation of the proposed fish 
counting system, and the usability testing are presented. 
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3.1. Requirement analysis 

In this section, the system requirement is presented. To gather the necessary requirement, a review 
of the existing systems and an interview with the targeted user have been conducted. From the activities, the 
following features should be included in the proposed system: 

- The system must be accessible on a desktop as well as on a mobile phone. In some cases, the fish 
hatchery worker may need to count the number of fish in the field, for example an ad hoc counting 
conducted at the fish tank; hence the system may be accessed using a mobile phone. 

- A module to enhance the presentation of the image. Due to illumination variations caused by the lighting 
conditions during the image acquisition, a mechanism to enhance the image presentation should be made 
available. 

A module whereby the user can access their past fish counting for record purposes. 


3.2. Fish counting model analysis 

In this section, the performance of the automatic fish counting model is analyzed. First, the analysis 
of the image enhancement conducted using CLAHE is presented. Second, the accuracy of the automatic fish 
counting model is described. The deepfish [26] dataset is used to generate and test the model. 


3.2.1. Image enhancement analysis 

To measure the effectiveness of CLAHE in enhancing the underwater image used in work described 
in this paper, a visual and statistical inspection is performed. Figure 4(a) is the original image and Figure 4(b) 
is an example of the enhanced image. As shown in Figure 4, CLAHE enhances the brightness of the 
foreground object without darkening the background in all cases, making the foreground more distinct. 
CLAHE makes all the fish objects more visible than the original image. This is important as the YOLOv5 
architecture, which is used to detect and count the fish, is based on object detection which requires higher 
edge visibility of the object of interest. 


Image 1 


Image 2 


Image 3 


Image 4 


Image 5 


Figure 4. Visual comparison of the image enhancement result: (a) original image and (b) the corresponding 
enhanced image using CLAHE 


To evaluate the effectiveness of image enhancement techniques, three common statistical metrics to 
assess the quality of an image were utlised: entropy, normalized root mean squared error (NRMSB), and 
absolute mean brightness error (AMBE). The optimal entropy range for improving image quality is between 
7.0 and 8.0 [27]. A lower NRMSE indicates a better contrast, which is desirable as it allows for better 
discrimination of the foreground from the background. An enhanced image with a low AMBE value shows 
less variations of its brightness compared to the original image. 
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Table 1 show the examples of the enhanced images produced performances that are close to the 
value of the optimal metrics as described in the foregoing paragraph. The statistical analysis is aligned with 
the analysis of visual inspection shown in Figure 4. Hence, the CLAHE has improved the underwater images, 
which is required before fish detection and counting can be performed effectively. 


Table 1. CLAHE performance for image enhancement 
Image Entropy NRMSE AMBE 


Image 1 7.47 0.70 0.03 
Image 2 7.31 0.57 0.08 
Image 3 7.36 0.78 0.02 
Image 4 6.72 0.41 0.17 
Image 5 7.10 0.50 0.20 


3.2.2. Fish detection and counting model analysis 

To generate the automatic fish detection and counting model, two public datasets were used, the 
FISH20 project [11] and the automatic zebrafish tracking project [28]. A total of 2132 images were used, 
whereby 80% were for training and 20% for testing. Fl-score is used to measure the performance. YOLOv5 
was deployed with Adam optimizer. Figure 5 shows the training loss of the model, the x-axis represents the 
number of epochs and the y-axis the loss value. 


Objectness 


== results 


Figure 5. Training loss of the model 


The model generates a bounding box when a fish is detected in the image. The box loss measures the 
errors of the boundary box locations and sizes prediction. The objectness loss measures the errors in predicting the 
box that contains a fish. Based on Figure 5, both box and objectness losses gradually decreased as the training 
epoch increased. The model generated by the 60 epochs produced the best training loss. The loss was increased 
when the epochs were greater than 60. Table 2 shows the fish counting Fl-score when applied to the test data, 
where the value was increased until when the epoch was set at 70. A higher Fl-score indicates that the system can 
detect as many fish as possible from the image. The results show that the proposed work is able to produce a good 
fish counting application, although more work is required to increase the performance. 


Table 2. Fish counting performance 
Epochs Precision Recall _ Fl-score 


20 0.87 0.11 0.20 
30 0.91 0.42 0.58 
40 0.93 0.48 0.64 
50 0.96 0.47 0.63 
60 0.99 0.66 0.79 
70 0.97 0.60 0.74 


3.3. Prototype demonstration 

In this section, the implementation of the prototype is presented, whereby some of the GUI of the system 
is described. Figure 6 presents the homepage of the proposed FiCo fish counting system. On this page, users are 
allowed to take a photo or upload an image to perform the fish counting. Once the image is uploaded, the fish 
counting module will commence after the user clicks the ‘calculate now’ button. 
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Figure 6. The homepage of the FiCO fish counting system 


After the fish counting is completed and the result is generated, the user will be redirected to the 
result page as shown in Figure 7(a). The result page shows the uploaded image, the resulting image whereby 
the fish in the image is identified and segmented, and the fish count. The generated result will be 
automatically saved to the database labelled with the user’s ID. The user can retrieve the past fish counting 
activities through the history of the fish counting page shown in Figure 7(b). The user can retrieve their fish 
counting result on this page whenever necessary. 


Try Another Photo 


(a) (b) 


Figure 7. The FiCo system pages for; (a) fish counting result page and (b) the history of the fish counting page 


3.4. Usability testing 

In the UT, participants were selected and given access to the prototype. Their feedback on the usability of 
the system was collected using an evaluation form. We tested the usability of the prototype at the UMS fish 
hatchery, and identified two subjects who were tasked to perform manual fish counting at the hatchery. Figure 8 
shows the UT rating of the prototype using the Likert scale of 1-5, whereby 5 being strongly agree. 

From Figure 8, the navigational, user-friendliness and instructions scored 3.5, which show that the 
system is easy to be used and navigated by a user that does not require comprehensive user training. The 
functionality score of 3 points indicates the system is providing sufficient basic features to meet the needs of 
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hatchery workers who are required to do fish counting. The design of the interface, however, was too 
simplistic and less appealing, with some fonts being inconsistently sized. This is reflected in the score 
obtained at 2.5. Based on the overall score of 3.2, the proposed system is acceptable but there is still room for 
improvement in order to increase its usefulness. 


Instruction 

User interface 
Functionality 
User-friendliness 


Navigational 


Overall rating 


Figure 8. The UT results 


3.5. Limitation and future work 

The presented fish counting system has several limitations. First, the dataset that was used to train 
the model for fish identification and counting was limited. Consequently, the model may not generalize well 
enough to provide a better estimation of fish detection in an image. Second, the system does not provide a 
feature to get the user’s feedback on the accuracy of the fish counting. Such a feature could be used as a 
gauge for how well they find the fish counting service provided. 

Some future work is identified. First, adding more image data with variations of data sources could prove 
useful in producing a more accurate model. Second, a mobile app version of the fish counting system would 
significantly improve its usability. The app would make it possible for the user to use the system anywhere in the 
hatchery and easily access their mobile device’s camera in order to take an image. This would also allow them to 
do so without interrupting their workflow and/or requiring them to leave equipment or tools behind. 


4. CONCLUSION 

In this paper, an intelligent fish counting system, FiCo, is presented. The images uploaded to the 
system are pre-processed using CLAHE. To identify and count the fish in the images, a deep learning 
architecture in the form of YOLOv5 is employed. The usability of the proposed FiCo is evaluated through its 
implementation at the fish hatchery in UMS. The results of the usability test indicate that FiCo is a viable 
solution that can aid hatchery workers in the labor-intensive task of fish counting, which is crucial for 
effective fish hatchery management. 
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